The present disclosure relates generally to computing systems, and more particularly to the inspection of system state during testing.
As the value and use of information continues to increase, individuals and businesses seek additional ways to process and store information. One option is a computing system. Computing systems may vary in complexity from a single processor operating in relative isolation to large networks of interconnected processors. The interconnected processors may be in close proximity to each other or separated by great distances both physically and as distance is measured in computer networking terms. The interconnected processors may also work together in a closely cooperative fashion or in a loose weakly coupled fashion. Because technology and processing needs and requirements may vary between different applications, the structure and arrangement of the computing system may vary significantly between two different computing systems. The flexibility in computing systems allows them to be configured for both specific users, specific uses, or for more general purposes. Computing system may also include a variety of hardware and software components that may be configured to process, store, and communicate information based on the needs of the users and the applications.
Additionally, some examples of computing systems include non-transient, tangible machine-readable media that include executable code that when run by one or more processors, may cause the one or more processors to perform the steps of methods described herein. Some common forms of machine readable media include, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, and/or any other medium from which a processor or computer is adapted to read.
Computing systems, whether they are complex servers or simpler standalone computers, tablets, or mobile devices, often include a significant number of interacting processors, hardware devices, and software components. The processors may include single processors, multi-core processors, and/or multiple processors. The hardware devices may include memory, secondary storage devices like flash drives and disk drives, and interfacing devices such as network interface components, graphics cards, hardware accelerators, and data acquisition systems, just to name a few. The software components may include operating systems, drivers for the hardware devices, services, application programming interfaces (APIs), software libraries, user applications, and/or the like.
To help ensure proper operation of the computing system one or more tests are typically designed and run on the computing systems to verify and/or validate the proper operation of the computing systems and the interactions between the processors, hardware devices, and/or software components. Before testing begins, the computing system is typically configured and/or provisioned with a baseline set of software components in a default and/or preferred configuration. A test, typically in the form of a software component, is executed to test one or more of the features of the pre-configured software components and/or interactions between the software components and hardware devices. The test may exercise features of the pre-configured software components and/or hardware and/or alter the configurations of the pre-configured software components and/or hardware. In some instances, the test may alter the configurations of the software components and/or hardware to test atypical and/or rarely used operation modes. Rather than test the entire computing system with a single test, a series of separate tests are typically developed to focus on specific subsets of the processors, hardware devices, and/or software components. Each of the separate tests may be designed and/or architected without knowing what other tests might be in use and so an assumption is made that the test will be run in a system with the baseline set of software components with a default configuration. This leads to the possibility that the changes made to the configurations of the software components and/or hardware devices made by other tests may interfere with and/or cause unexpected failures in later tests. This makes it difficult to determine whether a test failed due to a defect in the software components and/or hardware devices or whether the failure was caused by side effects from a previously run test. This may be further complicated when a change made by one test may not cause a problem until many tests later. One solution is to reset and/or rebuild the baseline set of software components and default and/or preferred configuration between each test, but this is typically very time consuming and impractical.
Accordingly, it would be desirable to provide systems and methods to make it easier to determine whether and to what extent tests are making changes to a configuration for a computing system.